A Machine Learning Approach for Identification Thesis and Conclusion Statements in Student Essays

This study describes and evaluates two essay-based discourse analysis systems that identify thesis and conclusion statements from student essays written on six different essay topics. Essays used to train and evaluate the systems were annotated by two human judges, according to a discourse annotation protocol. Using a machine learning approach, a number of discourse-related features were automatically extracted from a set of annotated training data. Using these features, two discourse analysis models were built using C5.0 with boosting: a topic-dependent and a topicindependent model. Both systems outperformed a positional algorithm. While the topic-dependent system showed somewhat higher performance, the topic-independent system showed similar results, indicating that a system can generalize to unseen data – that is, essay responses on topics that the system has not seen in training.